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Evaluation Systems in the Adult Education Program: 
The Role of Quality Indicators 



Like other publicly funded programs, adult education has faced increasing demands to 
demonstrate its effectiveness and the value of the instruction it offers. Over the last several years, 
amendments to the Adult Education Act, the federal legislation governing the adult education 
program, have strengthened accountability requirements. The 1988 reauthorization of the Act 
increased state requirements for local program evaluation by specifying six topic areas that 
evaluation should address and by mandating the use of standardized test scores in evaluation. The 
National Literacy Act of 1991 took a further step by requiring states to develop indicators of program 
quality within two years and to use them in evaluation of their local programs. The indicators were 
to assess programs’ success in recruitment, retention and improving students’ literacy skills. The 
Act also required the Department of Education to develop model indicators of program quality to 
guide states in the development of the indicators. 

Fulfilling its legislative mandate, the Division of Adult Education and Literacy (DAEL) 
published Model Indicators of Program Quality for Adult Education Programs in 1992 that 
presented eight quality indicators. Besides addressing the three required topic areas, the indicators 
described elements of quality for program planning, curriculum and instruction, staff development 
and support services. Within the next year, all states had revised their state plans to incorporate the 
quality indicators they had developed. 

Besides presenting the indicators themselves, DAEL's publication also provided a general 
framework to guide states’ development and use of the indicators. The framework distinguished 
indicators from measures and performances standards, and related them in a hierarchical, four-step 
process: 

1. Select topic areas to focus indicators. The National Literacy Act required indicators in 
recruitment, retention and literacy gains. However, states had the option to add other 
topics. 

2. Develop quality indicators in each topic area. A quality indicator was defined as a 
variable that reflects efficient and effective performance of the adult education program. 
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Evalvatios Systems in the adult Edvcatios Program: The Role of Ovautt Indicators 



Since their adoption, the quality indicators have become central to the program evaluation 
systems of most states. The indicators have helped states define the components of program quality 
and enabled them to develop measures for evaluating programs to ensure effective practice. This 
paper presents a sununary of state implementation of the quality indicators, focusing on the 
development of measures and standards for the indicators and the impact they have had on state 
accountability systems and program quality. The paper also discusses how states are using the 
quality indicators and presents a sununary of the indicator measurement systems in six states. 

Status of State Quality Indicator Measurement Systems 

The National Literacy Act stipulated only that states develop indicators of program quality in 
the areas of recruitment, retention and literacy gains. A review of the 1993 amendments to state 
plans, however, revealed that states developed indicators that were very similar, and in some cases 
identical, to the broader DAEL model indicators. All states have gone beyond the three required 
topics and developed indicators in areas of program planning, staff development and curriculum. 
Most states have also developed indicators of support services and a few states have indicators in 
such diverse areas as fiscal responsibility and facilities and materials. 

States also were required only to develop indicators of program quality - to complete the first 
two steps of the indicator framework. A review of state activities in this area in early 1996, however, 
shows that the states have adopted the DAEL framework fully and continued, or are still continuing, 
the process through the development of measures and performance standards. 

Table 1 shows state-by-state results of this review for 49 states, the District of Columbia and 
Puerto Rico and Figure 2 aggregates the findings across states. A majority of states have completed 
the indicator measures and standards development process. Almost two-thirds of the states have 
implemented measures of the quality indicators and about half of the states have accompanying 
performance standards to these measures. An additional 10 states have developed, but not yet 
implemented, the measures and 8 states have developed, but not yet implemented performance 
standards. All states have at least begun the development of measures and all but five states have at 
least begun the development of performance standards. 
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Evaluation Systems is the Adult Education Program: The Role of Ovauty Indicators 



Table 1 (continued) 

Status of Development of Measures 
and Standards for Quality Indicators 



state ' ' 


1 . , ■ Measures , | Standards 


|\ Beginning. | Oeveioped, Not { 1 Beginning j ' Develop^ Not ! 

1 Efforts j Implemented j imptemented | Efforts j Implement | iropisfiieiited 


AREA III 


Illinois 


1 I 

1 • 1 


• 1 1 


Indiana 


1 1 

' ' A 

• 


• 1 ! 

1 1 


Iowa 


' ' ^ 

• 


1 1 

j j • 


Kansas 


1 1 

1 1 « 




Michigan 


! ! a' 


1 ! 


Minnesota 


1 1 

• 1 ! 


1 1 

1 1 
1 1 


Missouri 


' 1 ... 

1 1 • 


1 1 

j j • 


Nebraska 


1 I 

' ' A 

• 


1 1 • 


North Dakota 


1 1 

j 1 « 


j j • 


Ohio 


1 1 « 


1 I • 


South Dakota 


I I 

1 I • 


1 1 • 


Wisconsin 


1 1 

I 1 ... 

! ! • 


L ! • 


AREA IV 


Alaska 


1 1 

1 1 « 


1 1 

j j • 


Arizona 


i i • 


1 i • 


California 


1 • 1 


• • ! 


Colorado 


1 1 

I 1 

1 I • 


j j • 


Hawaii 






Idaho 




• 1 1 


Montana 


1 1 

i i • 


1 1 

1 1 
1 1 


Nevada 


1 1 > 


1 1 • 


New Mexico 


• ! 1 


• 1 1 


Oregon 


1 1 


1 1 • 


Utah 


1 1 

1 1 > 


1 1 

1 1 


Washington 


1 1 • 


1 \ 

1 • 1 


Wyoming 


• 1 1 


1 1 

• ! 1 



’Measures and standards implemented on a pilot test basis. 

NOTE: Blank row indicates the state has not yet begun developing measures or standards. 
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Evaluation Systems in the adult Education Program: The Role of Quality Indicators 



■ To identify technical assistance needs. States can use measures and standards to identify 
local programs that need assistance and the areas where assistance is needed. 

■ For program improvement. By examining measures and standards for programs overall, 
the state can assess areas of strength and weakness in their delivery system and target weak 
areas for improvement. For example, measures could reveal that local programs are not 
meeting their recruitment targets, signaling the need for state redirection in this area. 

Table 2 shows how each state uses or plans to use indicator measures and standards and 
Figure 3 summarizes uses across states. Perhaps most surprising, 13 states currently use, and an 
additional 1 5 states plan to use, the measures and standards in program funding decisions. Local 
programs in these states could lose all or some of their state funding if they fall below indicator 
standards. States that use indicator measures in funding decisions give programs one to two years to 
correct problems identified through the measures and only terminate funding if the program’s 
problems continue past that time. The widespread use of the indicators for funding decisions 
demonstrates how seriously states have adopted the indicator system as a means of providing quality 
programming. 

As required by legislation, virtually all states use indicator measures to evaluate program 
effectiveness. Almost all states also use the indicators to promote program improvement and to 
identify technical assistance needs of local programs. 

State Implementation of the Indicators 

As with other aspects of the quality indicators, the National Literacy Act leaves 
implementation of the quality indicators to the states, specifying only that the indicators should be 
incorporated into states’ evaluation systems. The DAEL publication on the indicators also does not 
address implementation, but allows the states flexibility to use the process in the way that best meets 
their needs. To gain a more detailed understanding of how states have implemented indicator 
measures and standards, we contacted 10 of the 23 states that reported that they had fully 
implemented both measures and standards. The state directors in these states described how the 
indicators have been incorporated into the state’s evaluation system, how the state developed 
measures and standards, and their impact on state accountability systems. The state directors also 
offered their opinions on what made the process work in their states. 
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Table 2 (continued) 



Current and Planned Uses of Measures 
and Standards of Quality Indicators 



' state' 




.<X . ; ^ # 






.V ; .. A .V 

V ' • 




Use.Fon 










: ' Funding 




^;;■'■Programa^^-; ; 

, r Improvwmj^ ^ ; . 


'.a., technical ' 








recurrently 
Use ' 


1 Plan to - 

1 a 


.Currenttyr^^' 

" Use^-/;\ 




- iSJeff; 




Plan to : 
; Use : 




.r'^uee- -.-a 


:^?Planto ■ 

¥c,'-:u«i - 


Minnesota 


1 

1 

1 




• 




• 




• 


Missouri 


1 

• 


• 




• 




• 




Nebraska 


1 

1 


• 




• 




• 




North Dakota 


■ 

1 

1 


• 




• 




• 




Ohio 


I 

• 


• 




• 




• 




South Dakota 


• 


• 




• 






• 


Wisconsin 


i 

1 

1 


• 






• 


• 




AREA IV 


Alaska 


I 

• 


1 

• 




• 




1 

• 


1 


Arizona 


I 

• 


• 




• 




• 




California 


j 

1 

1 














Colorado 


• 


• 




• 




• 




Hawaii 


1 

1 

1 


• 




• 




• 




Idaho 


• 


• 




• 




• 




Montana 


1 

1 

1 




• 




• 


• 




Nevada 


1 

• 


• 




• 




• 




New Mexico 


1 

1 

1 














Oregon 


1 

1 

1 


• 




• 




• 




Utah 


1 

1 

1 


• 




• 




• 




Washington 


1 

i • 




• 




• 




• 


Wyoming 


1 

• 




• 






• 





NOTE: Blank row or column Indicates state will not use, or Is undecided about using, measures or standards for this purpose. 
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Evaluation Systems in the adult Education Program: The Role of Quauty Indica tors 



Some states also collect indicator measures through a separate report submitted annually by 
local programs and through the application process. In this latter method, local programs provide the 
measures in their application for funding and the state evaluates them on how well they meet the 
corresponding performance standards. 

Development of Measures and Standards 

The states contacted all developed measures and standards of the indicators in the same way: 
through the use of working groups and committees established for this purpose. In some states, the 
committees were subgroups of larger state committees working on broader state accountability and 
assessment issues. With few exceptions, the indicator committees were interagency, typically 
including representatives from labor, social service and vocational education agencies, literacy 
councils and workforce development boards. Some states also included local practitioners and 
business leaders. 

After developing draft measures and standards, the committees typically presented them to 
local program providers for comment and made revisions based on these comments. Several states 
then pilot tested the measures in a cross section of local sites before implementing the measures 
statewide. The usual time from development to implementation of the measures was one to two years. 

Impact on State Accountability Systems 

State directors uniformly noted that the measures and standards development process had a 
strong positive impact in their states. The main advantage identified was that the process raised 
awareness of program quality issues and gave state and local staff the opportunity to define and reach 
consensus on the characteristics of effective program operation. Measures and standards give 
direction and focus to program evaluation and provide programs with a way to evaluate themselves 
and work toward excellence, according to several state directors. By defining how they will be 
evaluated, the state also has made a statement on what programs should try to accomplish. 
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Conclusion; Unique Aspects of the Implementation of Quality Indicator Measures 

The implementation of indicator measures and standards is not unique to adult education 
programs. Most federally funded programs now have requirements for using quality measures and 
standards to demonstrate their effectiveness. For example, the most established and perhaps well 
known accountability system among federal programs is the performance standards system required 
by the Job Training Partnership Act (JTPA). Other programs that have formal performance measure 
requirements include vocational education programs, Food Stamp Employment and Training 
Programs and Job Opportunity and Basic Skills Programs. 

While the existence of formal accountability system is not unusual, the way the adult 
education quality indicator system has evolved distinguishes it in the following significant ways from 
similar initiatives. 

Voluntary implementation . Unlike the accountability systems of other programs, the 
implementation and use of measures and standards of quality indicators is not mandated by federal 
regulation. The only formal requirement is for states to develop indicators and incorporate them into 
the state’s evaluation procedures. On their own, states have adopted DAEL’s indicator framework to 
develop measures and standards and to use them as the basis for a formal system of program 
accountability. 

Focus onjrogram operation and instruction . Many accountability systems measure only 
participant outcomes. For example, JTPA performance standards assess participant employment, 
wages and employment retention. While the indicators in adult education also address student 
outcomes, other indicators address a wide range of program variables, including program planning, 
quality of curriculum and professional development. These indicators specifically identify elements 
of quality related to the content and operation of programs. Student outcomes are not ignored - 
DAEL’s model indicators include two indicators of student outcomes and most states have several 
measures of participant learning and advancement in the program - but adult education's specific 
focus on program elements in its accountability system is unique. This programmatic focus also goes 
beyond the legislative requirements for the indicators. 
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State Indicator Measurement Procedures 

■ Arkansas 

■ Colorado 

■ Connecticut 

■ Iowa 

■ Kentucky 

■ South Dakota 



APPENDIX 



ARKANSAS 

Arkansas incorporates measures and standards of the quality indicators as part of its local 
program monitoring process. A peer review team visits about one quarter of local programs 
annually and assesses program performance in program planning, administration, curriculum and 
facilities, staff development, recruitment, retention and educational gains. The program receives 
from zero to two points for each performance standard, except for student educational gains 
standards, where a scale of zero to four is used. After the monitoring visit, the state office sends 
a letter informing the program on where it stands in each area, and if below standard, areas of 
improvement needed. Although there is only one formal monitoring visit every four years, each 
program receives an annual follow-up visit from state staff . 

The state uses indicator measures to identify local programs that need technical assistance, 
to promote program improvement and to demonstrate program effectiveness. The measures are 
not tied to funding. 

Development of the Measures and Standards 



A state committee, consisting of state adult education staff, local program practitioners and 
state literacy council staff, developed the measures and standards for the indicators. Local 
programs across the state reviewed the initial draft of the measures and state and local staff pilot- 
tested the monitoring instrument in 26 local programs before it was finalized. After its first year 
of use, the state further refined the instrument. 

Impact on State Accountability System 

The indicator measures have improved the overall quality of local programs, according to 
the state adult education director. Programs now have better planning; improved administration, 
as measures by the quality of annual statistical information reported to the state; and better 
coordination with other agencies. State monitoring teams have also noticed improvement in 
program quality among community-based organization providers and in volunteer programs. 
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CURRICULUM. INSTRUCTION AND FACILITIES (Continued) 

CIF-Neasure 7: 

Various instructional activities and techniques are used. (Ill.b., 
III.c., IlI.e.) 

CIF-Standard 7; 



Dated evidence is available that the instructional staff uses a variety 
of instructional activities and techniques. ^Lesson plans, written 
observations, teacher logs, etc.) 

2 points - 2 or more types of documentation 
0 points - 1 or no types of documentation 



CIF-Neasure 8: 

Adequate instructional staff is available to meet student needs. 
(III. a., IV. a.) 

CIF-Standard 8; 



Dated evidence is available that administrators exercise flexibility in 
instructional assignments to meet student enrollment loads. (Attendance 
rosters, schedules, narratives, etc.l 

2 points - yes 
0 points - no 



CIF-Neasure 9: 

Instruction is adult-oriented. (Ill.d., Ill.f., Ill.g., Ill.h.) 
CIF-Standard 9; 

A variety o f adult-oriented instructional techniques, materials and 
supplies are utilized. 



2 points - yes 
0 points - no 
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CURRICULUM, INSTRUCTION AND FACILITIES (Continued) 

CIF-Heasure 13: 

The adult education facilities provide a safe and optimum learning 
environment. (VII. c.) 

CIF-Standard 13: 



Facilities include appropriate: 


a. 


classroom space for number of students 




work/study space for staff 


c. 


work/study space for administrators 




storage for materials and eouioment 



e^. furniture designed for adults 

fi equipment and furnishings to meet all program objectives 
and minimum standards 
Si restroom! si for both men and women 
II. safe parking area 

i . equipment is installed and functional 

i:. indoor and outdoor lighting 

2 points - all items (a-j) are met 
0 points - if any item is not met 



EDUCATIONAL GAINS (EG) 

Quality Indicator Ml: Learners advance in the instructional program or 

complete program educational requirements that enable them to continue their 
education or training. 

EG-Heasure 1: 

Students advance to a higher level of skill and/or 
competency. (IX. a.) 

EG-Standard 1: MAX I MUM 

POINTS 

Ll The percent of students (unduol icatedl with 40 or more hours 

of attendance who advance. jj^ 

Advancement rates for students enrolled for academic 
advancement and lifeskills advancement will be examined 
separately. Further, the advancement rates for students 
enrolling for academic studies will be determined for students 
entering on each of the three levels fO-5.9. 6-8.9. and 9-121 
because advancement is more difficult for students entering on 
lower grade level equivalent functioning levels. Scales A. B. 
and C will be used to examine advancement for students 
enrolled in an academic course of study. Scale D will be used 
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EDUCATIONAL GAINS (Continued) 



Scale D (life skill advancement) - may include ESI 

10-15 percent of students enrolled for lifeskills study 
make documented advancement 

16-20 percent of students enrolled for lifeskills study 
make documented advancement 

21-25 percent of students enrolled for lifeskills study 
make documented advancement 

26-30 percent of students enrolled for lifeskills study 
make documented advancement 

31 or more percent of students enrolled for lifeskills 
study make documented advancement 

NOTE: N/A's must be subtracted from total possible based on level 

of students served and kinds of classes taught. 



Points 

N/A 

01 

02 

03 

04 

05 



EG-Neasure 2: 

Students are referred to other programs which will continue to meet 
their educational needs. (II. d.) 

EG-Standard 2t 



There is dated documentation that information regarding the next 
level of education is provided to students who complete the highest 
level of study within a program year. (Referral loos, information 
packets, referral forms, etc.) 

2 points - yes 
0 points - no 

b. Documentation is available showing that providers who receive 
students out of their realm of services refer those students 
appropriately. 

2 points - documentation is available 
0 points - documentation is not available 



O 

ERIC 



15 



Quality Indicators 
Page 18 of 22 

STAFF DEVELOPMENT (SD) 



Quality Indicator: Program has an ongoing staff development process that 

considers the specific needs of its staff, offers training in the skills 
necessary to provide quality instruction, and includes opportunities for 
systematic practice and follow-up. 

SD-Neasure 1: 

Personnel possess required certification and/or training according to 
job duties assigned. (IV. a., b.) 

SD-Standard 1; 



Dated documentation exists that each director/coordinator, counselor, 
instructor/tutor, and oaraorofessional . etc., is trained and certified 
for his/her position as found in current "Adult Education Policy Manual" 
and "Qualification Standards for State Positions and Local Positions in 
Approved Programs" laporoved bv State Board of Vocational Education July 
1990 or newest edition). 

2 points - 100% 

0 points - less than 100% 



SD-Measure 2: 

All *new personnel participate in orientation. 
SD-Standard 2; 



Dated documentation exists that all new personnel have participated in 
an orientation process. (Calendars, agendas, attendance rosters, etc.l 

2 points - yes 
0 points - no 



SD-Measure 3: 

An annual staff development plan is implemented. (IV. b., IX. a., IX. b.) 
SD-Standard 3; 

li A dated list of recommendations for an annual staff development 

plan is implemented based on the following types of data: requests 

of personnel, examination of progress tests, retention rates, drop- 
out rates, GED test scores, literacy check-ups. follow-up surveys 
from community, industry and students. (Annual staff development 
schedules, agendas, source of needs, etc.) 

2 points - recommendations are implemented 
0 points - recommendations are not implemented 
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RECRUITMENT (Continued) 

RC‘Measure 2: 

An ongoing annual recruitment plan has been developed and implemented. 
RC-Standard 2; 

An annual rec ruitment j)l an exists which includes: a monthly calendar 
reflecting re cruitment activities and assigned personnel. Documentation 
is presented t hat activities were conducted. mat.Pd programs, speeches. 
thank-you let ters, evaluation of results of activities. <;tudent follow- 
up logs, etc.) 

2 points - plan developed and implemented 
0 points - plan not developed nor implemented 



RC-Measure 3: 

A variety of recruitment techniques are used. (VI. a., VIII. a.) 
RC-Standard 3: 



Categories and Technioues: 

Dated Audio Script 
radio 

television 

^ Dated Print 
posters 
fl vers 
newspapers 
displays 
inserts 

c. Dated Personal Contact 
letters 
telephone 
speakers 
county fairs 

2 points - 1 technique per month for each of the 12 months from 2 
or more categories 

1 point - 1 technique per month for each of the 12 months from 1 
category 

0 points - less than 1 technique per month for each of the 12 
months 



Quality Indicators 
Page 22 of 22 

RETENTION (Continued) 

RT -Measure 4: 

Students accrue between twelve and forty hours of instruction within a 
program year. (Vl.b.) 

RT-Standard 

Students accrue between twelve and forty hours of instruction within a 
program year. 

2 points - 35 - 50% of all students 
1 point - 20 - 34% of all students 
0 points - 19% or less of all students 



RT-Measure 5: 

Students accrue forty or more hours of instruction within a program 
year. (Vl.b.) 

RT-Standard 5; 

Students accrue forty or more hours of instruction within a program 
year. 



2 points - 30 - 40% of all students 
1 point - 20 - 29% of all students 
0 points - 19% or less of all students 

RT-Measure 6: 

Ten percent of absentee students return to the program. (Vl.b.) 
RT-Standard 6: 



Students return to the program after being absent for one monthly 
reporting period. 

2 points - 10% or more 
1 point - 5 - 9% 

0 points - 0-4% 
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COLORADO 

Colorado collects indicator measures in three ways: through . its monitoring procedures, on 
the program application, and through an annual report. Peer review teams use the state’s local 
monitoring instrument, which was recently revised to collect indicator measures, in their annual 
review of approximately one-third of local programs. The instrument includes measures of 
educational gains, program planning, curriculum and instruction, staff development, support 
services, recruitment, retention, administration and facilities. On each performance standard, the 
program is assessed according to whether it does or does not meet the standard, exceeds the 
standard, or whether the standard does not exist for the program. Some of the more stable 
program measures, such as program planning, are also addressed in the program’s application for 
funding. At the end of each program year, local programs submit a report that addresses the 
student outcome and recruitment measures. 

Besides using the indicators for program effectiveness, program improvement and to 
identify technical assistance needs, the state uses indicator measures in its funding decisions. If a 
program fails to meet performance standards, it is given a year to improve. If the program fails 
to improve during that time, it can no longer receive state funds. The main problem programs 
have had in meeting standards have been in documenting student progress. 

Development of the Measures and Standards 

A task force of state staff, local program directors and teachers reviewed the indicators and 
developed draft measures. The program directors and teachers then presented these measures to 
their programs for comment and the full committee then revised the measures. The task force 
presented the final measures to all programs at the annual state conference. The development 
process lasted about 18 months. 

Impact on State Accountability System 

The main benefit of the indicator measurement system is that local programs are now well 
versed in program accountability issues. Because of their participation in this process, program 
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APPENDIX 



CONNECTICUT 

Connecticut’s indicator measures are incorporated into its program monitoring instrument. 
Local programs first complete the monitoring form themselves, and a multi-agency team then 
reviews programs annually. The measures in the instrument assess the program in the areas of 
program planning and operations, recruitment, retention, educational gains, curriculum and 
instruction, support services and staff development. Some of the educational gains and retention 
measures - test scores and attendance - are taken from the state’s management information system 
for each program. 

The state uses the indicators to demonstrate program effectiveness to other agencies and 
audiences within the state and for program improvement. If a program falls below standards, the 
state targets technical assistance to the program in the deficient area. If the program does not 
make a good faith effort to correct its problems, funding can be reduced or eliminated. 
Connecticut’s two year funding cycle for local programs strengthens the state ability to affect 
changes. 

Development of the Measures and Standards 

An interagency committee with representatives from labor, social services, local school 
districts, the workforce development board, business and state education staff developed the 
measures and standards for the indicators. A draft of the measures and standards was presented to 
local programs for comment prior to completion. 

Impact on State Accountability System 

The indicator measures and standards development process has served the state well by 
clearly defining the expectations for program quality and systematizing the state’s evaluation 
activities. An added advantage is that local programs are now better prepared to describe their 
activities and report outcomes to outside agencies. Programs are better prepared for interagency 
collaboration, which will become increasingly important in the current programmatic environment. 
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IOWA 

Iowa translated many of its quality indicators into benchmarks to assess whether the state’s 
15 community colleges, which provide adult education instruction, are achieving long-range 
strategic goals. The state established a total of 29 benchmarks in the areas of educational gains, 
program planning, curriculum and instruction, staff development, support services and 
recruitment/retention. Each benchmark describes a measure and a target for the year 2000 and a 
second target for 2005. For example, for educational gains, one benchmark is the “Percentage of 
adults 18 years and over who have attained a high school or equivalent diploma: 85% for 2000 
and 90% for 2005.” The state designated 16 of the benchmarks as “core” benchmarks to identify 
the basic values inherent in the adult basic education program. The benchmarks will be used to 
guide program policy and priorities, demonstrate program effectiveness and quality and to identify 
areas needing continued improvement through technical assistance. 

The state will obtain the benchmark measures from multiple sources. Student educational 
gains, for example, will be drawn from the state’s management information system. Many of the 
program measures, such as for the planning process, will come from program monitoring. 

Broader measures, such as the overall literacy levels in the state, will require the state to conduct 
research studies to assess progress. The benchmark system has just taken effect in 1996 and will 
be monitored annually by the state education office. 

Development of the Benchmarks 

The state used the same committee that developed the quality indicators, measures and 
standards to develop the benchmarks. The committee, was composed of state education staff and 
the basic education coordinators of the state’s community colleges. Separate subcommittees 
worked on each benchmark. 

Impact on State Accountability System 

With its development of benchmarks, the adult education program is at the forefront of the 
program accountability process in Iowa. The benchmarks clearly communicate to other agencies 
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APPESDIX 



KENTUCKY 

Kentucky measures its indicators of program quality in two ways: through a self- 
evaluation that each local program performs annually and more formally through a program 
compliance review, conducted annually on one-third of programs by peer review teams. Indicator 
measures and standards assess educational gains, program planning and evaluation, curriculum, 
instruction and instructional setting, professional development, support services, recruitment and 
retention using a three point scale of excellent, satisfactory or needs improvement. 

Programs that fall below standards must develop a program improvement plan and are 
provided technical assistance from state staff to implement the plan. If the program’s subsequent 
improvement is not satisfactory, the program can lose its state funding. The state also uses the 
measures and standards to identify strengths and weaknesses in the state delivery system. 

Development of the Measures and Standards 

A committee of practitioners, staff from community organizations, state education staff 
and state literacy personnel developed Kentucky’s indicators measures and standards. The 
committee first established the indicators and distributed them to local programs. About a year 
later, the measures and standards were developed and implemented. 

Impact on State Accountability System 

The quality indicators have given direction and focus to the state and local programs in 
defining the goals and purposes of the adult education program. The measures and standards have 
helped the state identify and focus technical assistance activities to programs to improve the 
statewide system. The local programs can use the indicators as a tool to evaluate themselves and 
move toward excellence. 
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Sample Measures • Documented recruitment effort that correlates proportionately to the targeted 

populations in the community/area served (percentage of targeted populations 

, ,, . f M , may be identified using 1990 Census data) 
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Appendix 



SOUTH DAKOTA 

South Dakota developed a self-assessment instrument to measure quality indicator 
measures and standards. All local ABE programs complete the self-assessment annually. The 
state also conducts an onsite review of a random sample of 20 percent of local programs annually. 
The self-assessment addresses the indicators for educational gains, program planning, curriculum 
and instruction, staff development, recruitment and retention. For each performance standard, 
programs indicate whether they achieved the standard, the plans they have to improve quality, the 
timeline for making improvements, the person responsible for making the improvements and the 
technical assistance they will need to achieve their goal. 

Programs are not required to address all of the standards, but select the areas where they 
want to develop their program further. Local programs can then request technical assistance from 
the state, which may be provided through inservice training, staff development training or visits 
from the state office. Although currently the state uses the measures and standards as a program 
improvement and technical assistance tool, funding decisions may be based on program and 
student progress measures in the future. 

Development of the Measures and Standards 

A state work group composed of the state adult education director; local practitioners; 
representatives from labor, vocational education and higher education; and the directors of the 
state literacy council and lifelong learning council developed the measures and standards, as well 
as the self assessment instrument. The development process took about a year. 

Impact on State Accountability System 

The major benefit of the measures and standards is that it allows program staff to select 
the areas where they want their program to improve. The self assessment instrument then gives 
them the tool for understanding how to make the improvements. The process also makes 
programs aware that technical assistance is available and encourages them to view the state office 
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